Colorization is the art of adding color to a monochrome image or movie. The idea of coloring photos and films is not new. Hand coloring of photographs is as old as photography itself. There exists such examples from 1842 and possibly earlier. It was practiced in motion pictures in the early 1900s by the French Company Pathé, where many films were colored by hand. It was widely practiced also for filmstrips into the 1930s. A computer-assisted process was introduced in 1970 for adding colors to black and white movies.
Various early computer-based colorization techniques include straightforward approaches such as luminance keying. This method uses a user-defined look-up table which transforms gray-scale into color. Others extended this idea by matching luminance and texture rather than just the gray-scale values.
Manual segmentation has been employed to divide a gray-scale image into a set of layers and then to estimate an alpha channel using Bayesian image matting. The final image is constructed using alpha-blending. More recently, a segmentation method has been optimized for colorization of black-and-white cartoons.
Other approaches assume that homogeneity in the gray-scale image indicates homogeneity in the color. In other words, the geometry of the image is provided by the geometry of the gray-scale information. Often, in addition to the gray-scale data, color hints are provided by the user via scribbles in these methods. Probabilistic relaxation methods have been employed. Another researcher has solved an optimization problem that minimizes a quadratic cost function of the color difference between a pixel and its weighted average neighborhood colors. Another has proposed “inpainting” colors constrained by the gray-scale gradients and color scribbles that serve as boundary conditions. This method involves solving linear or non-linear Poisson equations.
The main shortcoming of these previous approaches is the intensive computational cost needed to obtain good-quality results. Some have addressed this issue with a faster algorithm that propagates colored seed pixels in all directions, where the coloring is done by choosing from a preselected list of color candidates. However, this method produces visible artifacts of block distortion, since no color blending is performed.
In addition to colorizing monochrome images, needs exist for recolorizing images that already contain color information. For example, advertisements may wish to emphasize products by showing a scene where only the product itself is in color. Sepia and related effects may create striking visual presentations. Online sales outlets may desire to allow a customer to visualize automobiles or clothing in different colors or styles on Web pages. Photographers may wish to selectively blend new colors with existing colors while retouching images. Beyond merely adding or replacing colors, photographers and artists commonly employ a number of effects or filters for manipulating images for various purposes.
Many of these applications are much more useful when images can be colored or manipulated accurately, with only a small input from the user, and quickly. Some applications are made practical only when the manipulation can be performed interactively in near real time—that is, within a time interval short enough that users can feasibly wait for the results and perhaps provide changes or comparisons without having to interrupt their thoughts or move to another task.
Video images as well as still may profit from such manipulation, especially where the user inputs could be applied to only a relatively small number of frames in a sequence. In the past, however, colorizing video images has required complex and slow techniques such as optical flow, which calculates a velocity field that warps one image into another, usually very similar, image in a sequence. Besides being slow in itself, optical flows generally require execution of an algorithm in addition to already slow image-manipulation algorithm.